Effects of Occam's Razor in Evolving Sigma-Pi Neural Nets

Several evolutionary algorithms make use of hierarchical representations of variable size rather than linear strings of xed length. Variable complexity of the structures provides an additional representa-tional power which may widen the application domain of evolutionary algorithms. The price for this is, however, that the search space is open-ended and solutions may grow to arbitrarily large size. In this paper we study the eeects of structural complexity of the solutions on their generalization performance by analyzing the tness landscape of sigma-pi neural networks. The analysis suggests that smaller networks achieve, on average, better generalization accuracy than larger ones, thus connrming the usefulness of Occam's razor. A simple method for implementing the Occam's razor principle is described and shown to be eeective in improving the generalization accuracy without limiting their learning capacity.